6 research outputs found
Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects
We develop a method that recovers the surface, materials, and illumination of
a scene from its posed multi-view images. In contrast to prior work, it does
not require any additional data and can handle glossy objects or bright
lighting. It is a progressive inverse rendering approach, which consists of
three stages. First, we reconstruct the scene radiance and signed distance
function (SDF) with our novel regularization strategy for specular reflections.
Our approach considers both the diffuse and specular colors, which allows for
handling complex view-dependent lighting effects for surface reconstruction.
Second, we distill light visibility and indirect illumination from the learned
SDF and radiance field using learnable mapping functions. Third, we design a
method for estimating the ratio of incoming direct light represented via
Spherical Gaussians reflected in a specular manner and then reconstruct the
materials and direct illumination of the scene. Experimental results
demonstrate that the proposed method outperforms the current state-of-the-art
in recovering surfaces, materials, and lighting without relying on any
additional data.Comment: 12 pages, 10 figures. Project page:
https://authors-hub.github.io/Factored-Neu
Unpaired Depth Super-Resolution in the Wild
Depth maps captured with commodity sensors are often of low quality and
resolution; these maps need to be enhanced to be used in many applications.
State-of-the-art data-driven methods of depth map super-resolution rely on
registered pairs of low- and high-resolution depth maps of the same scenes.
Acquisition of real-world paired data requires specialized setups. Another
alternative, generating low-resolution maps from high-resolution maps by
subsampling, adding noise and other artificial degradation methods, does not
fully capture the characteristics of real-world low-resolution images. As a
consequence, supervised learning methods trained on such artificial paired data
may not perform well on real-world low-resolution inputs. We consider an
approach to depth super-resolution based on learning from unpaired data. While
many techniques for unpaired image-to-image translation have been proposed,
most fail to deliver effective hole-filling or reconstruct accurate surfaces
using depth maps. We propose an unpaired learning method for depth
super-resolution, which is based on a learnable degradation model, enhancement
component and surface normal estimates as features to produce more accurate
depth maps. We propose a benchmark for unpaired depth SR and demonstrate that
our method outperforms existing unpaired methods and performs on par with
paired
Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds
Constructing high-quality generative models for 3D shapes is a fundamental
task in computer vision with diverse applications in geometry processing,
engineering, and design. Despite the recent progress in deep generative
modelling, synthesis of finely detailed 3D surfaces, such as high-resolution
point clouds, from scratch has not been achieved with existing approaches. In
this work, we propose to employ the latent-space Laplacian pyramid
representation within a hierarchical generative model for 3D point clouds. We
combine the recently proposed latent-space GAN and Laplacian GAN architectures
to form a multi-scale model capable of generating 3D point clouds at increasing
levels of detail. Our evaluation demonstrates that our model outperforms the
existing generative models for 3D point clouds
Multi-sensor large-scale dataset for multi-view 3D reconstruction
We present a new multi-sensor dataset for 3D surface reconstruction. It
includes registered RGB and depth data from sensors of different resolutions
and modalities: smartphones, Intel RealSense, Microsoft Kinect, industrial
cameras, and structured-light scanner. The data for each scene is obtained
under a large number of lighting conditions, and the scenes are selected to
emphasize a diverse set of material properties challenging for existing
algorithms. In the acquisition process, we aimed to maximize high-resolution
depth data quality for challenging cases, to provide reliable ground truth for
learning algorithms. Overall, we provide over 1.4 million images of 110
different scenes acquired at 14 lighting conditions from 100 viewing
directions. We expect our dataset will be useful for evaluation and training of
3D reconstruction algorithms of different types and for other related tasks.
Our dataset and accompanying software will be available online